Changes in the Review Process
Marketing Science (Spring 2002, Vol. 21, No. 2)

Electronic Publishing and Marketing Science

Anonymity and The Review Process at Marketing Science

Steven M. Shugan
Editor-in-Chief, Marketing Science
Russell Berrie Eminent Scholar Chair and Professor of Marketing
Warrington College of Business Administration
University of Florida
201 Bryan Hall 
Campus Box 117155
Gainesville, FL 32611-7155
Tele: 352-392-0161 x 1236
Fax: 352-846-0457
MarketingScience@cba.ufl.edu

1. Electronic Publishing and Marketing Science

We are changing to a completely electronic review process at Marketing Science. An electronic review process is now possible because of recent technological advances. The process is possible because of both widespread high speed Internet access and the widespread adoption of Adobe Portable Document Format (PDF). PDF files allow automatic file compression, file concatenation and, more importantly, manuscripts to have identical appearance when viewed on almost any computer. Previous incompatibilities between different versions of Microsoft Word, for example, may cause manuscript appearance (particularly equations) to vary from computer to computer depending on many factors (including installed fonts).

The authors, reviewers, editor and area editors will correspond by e-mail. Participants in the review process, who require particular documents from the editor, will be able to download those documents from our web server. The editor will provide secure links to these required documents. Hence, we will avoid sending e-mail messages with large attachments.

Obviously, there will be initial costs. There are at least four costs. First, everyone involved in the review process must make some effort to be compatible with our system. Second, changing the system will create transaction costs (especially for manuscripts in process). Third, administering the new system will require more skills and knowledge of new technologies.  Fourth, reviewers who want paper copies will need to print them.  (Of course, high-speed printers are becoming common.)

Despite these costs, the potential benefits are overwhelming. It becomes far easier to involve European and Asian researchers in our process. In the past, shipping delays and lost manuscripts caused substantial delays in the processing of several manuscripts. Remember that, at any time, the review process works at the speed of the slowest reviewer. One lost document in transit or by an overseas reviewer can delay the process several weeks.

Another advantage is that an electronic review process makes record keeping easier and more accurate. We get instant acknowledgement that reviewers and area editors received documents. From e-mail records, we know the date and time when each person in the process both received and sent documents.

Still another advantage is a reduction in the total time for the review process. Authors can send manuscripts within minutes. After reviewer assignment, reviewers get manuscripts within minutes. As reviews arrive, they are combined and area editors can monitor reviews as they arrive. After the area editor report is complete, the editor gets the area editor report within minutes. After the editor makes a decision, the authors learn the decision and get the supporting documents within minutes. Moreover, we can reach all participants in the process including those who have changed their residence, are traveling or are merely working at different locations. Beyond these benefits for the average manuscript, the electronic process eliminates some major delays associated with problem manuscripts. These include, the reviewer declining to review a manuscript and returning it, a manuscript being lost in the mail, errors in reproduction (e.g., missing or illegible pages), the need to obtain additional copies of the documents (e.g., manuscripts, reviews, reports) and problems occurring during holiday periods.

The electronic process may allow us some new luxuries. For example, the editor and an AE in Israel may now discuss reviewer selection without waiting two weeks for delivery of the manuscript.

The electronic process should eventually lead to more automatic functions and greater reliability. It should also lead to physical space savings and lower out-of-pocket costs (e.g., express mail, duplication, record-keeping, package safety). Finally, the whole system is more environmentally friendly (e.g., saving energy and trees). We now turn to the second change in the review process.

2. Anonymity and The Review Process at Marketing Science

It is necessary to adopt some basic policies regarding anonymity in the review process at Marketing Science. We will be unable to resolve all the issues here, but this letter outlines some thoughts on this area and a policy for Marketing Science.

Obviously, there is a great deal of discussion about the peer-review process and a large extant literature on the process. That literature is fascinating (see my last editorial for citations).

In sum, it appears that the peer review system assumes that an editor lacks sufficient expertise (and time) to evaluate every manuscript. The primary responsibility of the editor becomes locating reviewers with the required expertise and approving articles after they undergo peer review. The reviewers evaluate the manuscripts.

INFORMS policy is to have double-blind reviews to provide a fairer process for authors. Our purpose here is not to debate that policy but explore how to implement it. We now promise anonymity to both reviewers and authors. One justification for this policy is that anonymous reviewers have better incentives to participate and provide objective reviews (without fear of lobbying from authors or possible retribution). Anonymous authors get more objective reviews based on the quality of their research rather than their affiliation or pedigree. Of course, some authors argue that reviewer anonymity decreases accountability. Some reviewers argue that author anonymity decreases available information. Hence, as usual, there is a compromise between costs and benefits. A double-blind review system seems to suggest that the compromise favors accountability for the editor (who makes decisions) and objectivity for evaluators.

The problem is how to implement a double-blind system. There are technical problems in implementing the system in an electronic age when authors can easily reveal their identities. However, there is very little we can do about that. We can only control when we release information about identities.

Obviously we fail to promise complete anonymity because editors know all identities. So we ask what it means to promise anonymity. My interpretation is that when we promise people anonymity, we promise to only reveal information about their identities to only necessary pre-specified individuals and no one else. To me, it follows that we must inform anonymous reviewers about who knows their identities. It also follows that we must inform anonymous authors about who knows their identities.

The real problem concerns the area editor (AE). Some past editors of Marketing Science have promised anonymity to the AE and some have not. My feeling is that the AE is an evaluator who possesses more specialized expertise than the editor. Given that our manuscripts often depend on specialized knowledge from diverse fundamental disciplines, the AE evaluation is invaluable. Moreover, in the current system, the AE contribution to quality control and improving manuscripts seems exemplary. Ergo, the AE should have the privilege of anonymity to encourage objective evaluations of the manuscript given the reviews. That privilege may encourage more relatively junior researchers to become area editors.

In contrast, if the AE were a decision-maker, who assumes responsibilities in order to decrease the workload of the editor, anonymity becomes less appropriate and accountability trumps objectivity. However, the current editor lacks this justification to delegate decision-making authority to the AE. The delegation of decision-making is unnecessary because the number of manuscripts in process currently is manageable (on average, less than on paper per AE per month). Moreover, the current editor seeks to provide a more uniform process across manuscripts. At this point, although the AE’s recommendation is of utmost importance, the AE is a source of specialized expertise and more of an evaluator than a decision-maker. Hence, the policy is to keep the AE anonymous to authors.

We next ask whether an anonymous AE should know the identities of authors and reviewers. In a truly double blind system (whether the AE is anonymous or not), the answer is clearly no. We promise anonymity to both the reviewers and the authors. We break that promise by revealing their identities to anyone, and in particular, individuals unknown to them. My conclusion is that the past system was, de facto, not truly double blind because we revealed author and reviewer identities to the anonymous AE.

However, the justification for reviewer anonymity (e.g., possible author retribution or lobbying) seems inappropriate when applied to the AE. The authors have little to fear from the AE. Moreover, to properly interpret and synthesize the reviews, the AE needs to know the identities of the reviewers and the expertise each brings to the process. Although knowing identities always creates biases, the value of the information overwhelms the potential bias. Moreover, given an asymmetric loss function, some bias is optimal. Hence, it seems both critical and justifiable that the AE knows the identities of the reviewers. Hence, revealing the reviewer’s identities to the AE may remain consistent with a double-blind process.

Given that conclusion, I believe that the reviewers need to know the AE identities. The reason is that we have promised the reviewers anonymity. Given that promise, we are obligated to tell the reviewers, at the outset, who will know their identities. It is unfair to reviewers to reveal their identities to individuals unknown to them, after having promised them anonymity. Hence, the new policy is to reveal AE identities to reviewers.

Next, we come to author’s identities. Here, I feel that when we promise anonymity to authors, we should avoid revealing their identities to anyone, and in particular anonymous evaluators (reviewers or AE). That seems consistent with the promise of a double-blind review process. Otherwise, the process fails to be double blind. Hence, the new policy is to keep the author’s identity anonymous to everyone except the editor (although complete anonymity may also be possible).

Finally, at the time of publication, all participants retain the right of anonymity but also have the right to waive it. The authors will certainly want to reveal their identities. The AE’s have the option of revealing their identities or not. The policy is voluntary disclosure.

©2002 University of Florida